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Abstract 

We define some appropriate statistical quantities that indicate the physical 
significance (reliability) of confidence intervals in the framework of both Fre- 
quentist and Bayesian statistical theories. We consider the expectation value 
of the upper limit in the absence of a signal (that we propose to call "exclu- 
sion potential" , instead of "sensitivity" as done by Feldman and Cousins) and 
its standard deviation, we define the "Pull" of a null result, expressing the 
reliability of an experimental upper limit, and the "upper and lower detection 
functions" , that give information on the possible outcome of an experiment if 
there is a signal. We also give a new appropriate definition of "sensitivity", 
that quantifies the capability of an experiment to reveal the signal that is 
searched for at the given confidence level. 

PACS numbers: 06.20.Dk 
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I. INTRODUCTION 



The possibility to apply successfully Frequentist statistics to problematic cases in frontier 
research has received a fundamental contribution with the proposal of the Unified Approach 
by Feldman and Cousins JIJ. The Unified Approach uses Neyman's method || to construct a 
confidence belt that guarantees the derivation of confidence intervals with a correct Frequen- 
tist coverage (see [§-|5[]). Furthermore, the Unified Approach allows to obtain an automatic 
transition from two-sided confidence intervals to upper limits in the case of negative results, 
preserving the property of a correct Frequentist coverage. Several works |6|- |T0|1 have followed 
the Unified Approach paper discussing alternative methods to construct the confidence belt. 
Hence, at present several Frequentist methods with interesting properties are available and 
under discussion. Their performances are also often compared with those of the Bayesian 
Theory (see Ref. 0). 

In this paper we define some appropriate statistical quantities that could help to decide 
which is the most appropriate method to use in order to obtain confidence intervals with 
the desired level of physical significance (see also Ref. [0). We also define the sensitivity 
of an experiment to a signal. 

The physical significance of a confidence interval is its degree of reliability, which is 
very important, because scientists use confidence intervals provided by experiments or spe- 
cialized compilations (for example, the Review of Particle Physics as inputs for their 



calculations. Unreliable confidence intervals may lead to wrong or hazardous conclusions. 

In Section |TT| we consider the expectation value of the upper limit in absence of a signal 
(that we propose to call "exclusion potential") and its standard deviation. In Sections |TXT| and 
|TV| we discuss the possibility to calculate the expectation value ( "upper and lower detection 
functions") and the standard deviation of the upper and lower limits of the confidence 
interval produced by an experiment in the presence of a signal and we propose an appropriate 
definition of the "sensitivity" of an experiment to the signal searched for. 

In this article we consider explicitly the case of a Poisson process with known background 
in the framework of Frequentist and Bayesian statistical theories, but similar considerations 
apply also to the case of a Gaussian distribution with boundary. 



II. EXCLUSION POTENTIAL 

An important quantity introduced by Feldman and Cousins []J is the average upper 
limit for the signal \x that would be obtained by an ensemble of experiments if = (in the 
case of a Poisson process with known background, "the average upper limit that would be 
obtained by an ensemble of experiments with the expected background and no true signal"). 
They called this quantity "sensitivity", but we think that this name is quite misleading, 
because it gives the impression that this quantity represents the expected capability of the 
experiment to reveal a true signal []. Instead, the true signal is assumed to be absent. 



For example, one can check on the on-line Webster Dictionary at bttp: / /www.m 



w. com/dictionary that "sensitivity" is "the quality or state of being sensitive" and the adjective 



Hence, it is clear that the quantity under consideration does not represent the sensitivity of 
the experiment to the signal that is searched for, but it represents the expected upper limit 
for /i that will be obtained if there is no signal. Therefore, we propose to call this quantity 
"exclusion potential'^, a name that we will use in the following. As a further justification 
of this name, we note that in the case of neutrino oscillation experiments the exclusion 
potential is associated with the so-called "exclusion curves" in the space of the neutrino 
mixing parameters. 

In the case of a Poisson process with known background the exclusion potential is given 

by 

oo 

li cp (b, a) = ^2 A*u P (n, b, a) P(n\fi = 0, b) , (2.1) 

n=0 

where n is the number of counts, b is the expected mean background, \i is the mean true 
signal, P(n\n, b) is the Poisson p.d.f. for the process and fJ, up (n, b, a) is the upper limit of the 
100(1 — a)% confidence interval for \i corresponding to n counts. The exclusion potential 
Hep depends on the values of the upper limits fi np (n, b, a), which are different in the different 
methods for calculating the confidence belt. 

It is interesting to note that the above definition of exclusion potential can be extended 



to the results obtained with the Bayesian Theory (see, for example, [14|) if /x up is interpreted 
as the upper limit of the Bayesian credibility interval for /x. In the following we will consider 
the Bayesian Theory assuming a flat prior and shortest credibility intervals for /i. In this 
case the posterior p.d.f. for /i is 

P(/i\n, b) = (b + nY e-" U , (2.2) 

and the probability (degree of belief) that the true value of /i lies in the range [/ii,/i2] is 
given by 

P(„ 6 K „]h t) = («- ± - «- ± £±jp) (t f) " . (2.3) 

\ k=0 k=0 / \k=0 J 

The shortest 100(1 — a)% credibility intervals [/ii ow , A*up] are obtained by choosing /ii ow and 
/i up such that P(/i G [/ii ow , /j up ] \n, b) — 1 — a and P(fi\ ow \n, b) = P(/i up \n, b) if possible (with 
/"low > 0), or /i low = 0. 



"sensitive" means "capable of being stimulated or excited by external agents (as light, gravity, or 
contact)". In our case, since the background is known, it can be considered an "internal agent", 
whereas the true signal is the "external agent" under investigation. Prom the physical point of 
view the sensitivity is related in many fields of application to the concept of minimum detectable 
signal, therefore it is a quantity defined for u > 0, not when [i = 0. 
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The on-line Webster Dictionary at http: / / www.m-w.com / dictionary says that "potential" is 



"something that can develop or become actual" . 
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The solid lines in Figs. [I] and |^ represent the exclusion potential as a function of the 
background in the interval < b < 20 for a confidence level 90% (a = 0.10) obtained 
with the Unified Approach |]J (Figs. |1|A and §A), with the Bayesian Ordering method]] 
H (Figs. |TjB and |3|B) and with the Bayesian Theory assuming a flat prior and shortest 
credibility intervals (Figs. [I]C and|3|C). 

Feldman and Cousins |l| suggested that in the cases in which the measurement is less 
than the estimated mean background and a stringent upper bound on /i is inferred, the 
experimental collaboration should report also the exclusion potential (that they call "sensi- 
tivity") of the experiment. This is also recommended by the Particle Data Group [|T3]]. 

In practice, the comparison of the upper bound and the exclusion potential is used as 
an assessment of the reliability of the upper bound. However, one can notice that a simple 
comparison of the upper bound with the exclusion potential does not give any information 
unless a meaningful scale of comparison is given. We think that a meaningful scale is the 
possible fluctuation of the upper bound /Ai P (n, b, a) in an ensemble of experiments with the 
expected background and no true signal. A quantification of this fluctuation is provided by 
the standard deviation <r ep (b, a) of the upper limit fi up calculated assuming fi = 0, 

oo 

a 2 p (b, a) = ^ K P (™, b, a) - /i ep (b, a)} 2 P(n\fi = 0, b) . (2.4) 

71=0 

The shadowed regions delimited by the dashed lines in Fig. [l] represent the range [/i ep — 
o"e P ,/%, + a ep ] for a = 0.10 obtained with the Unified Approach (A), with the Bayesian 
Ordering method (B) and with the Bayesian Theory assuming a flat prior and shortest 
credibility intervals. The corresponding probability to obtain an upper bound in the interval 
[/i ep — 0epi /i ep + a ep ] if /i = is shown in the three upper figures |l]a, [L]b and [T]c. One can see 
that, except for small values of the background b, the probability to obtain an upper bound 
in the interval [/i ep — cr ep ,/%> + a e P ] if /i = is not far from 68% in the three considered 
methods. The probability curves in Fig. [I] have wild jumps because n is an integer and /i up 
has discrete jumps as n is varied. 

As an illustration, in Fig. |2| we have plotted the probability of the possible values of the 
90% CL upper bound /i up if /x = and b = 13. One can see that the upper bound /x up 
can assume only discrete values. The probability to obtain an upper bound in the interval 
[/i ep — cT ep ,/i ep + o~cp], delimited by the dotted vertical lines in Fig. ^, has discrete jumps as 
b is changed, depending on which possible values of the upper bound are included in the 
interval. 

It is also possible to calculate the possible range of fluctuation of the upper bound /^up 
with a desired probability 7. The shadowed regions delimited by the dashed lines in Fig. |3| 
show the 90% width of the 90% CL upper limit, i.e. the possible range of fluctuation of 
the /U up (n, b, a — 0.90) with probability 7 = 0.90 as a function of b if /i = 0. This range of 



3 The Bayesian Ordering method, as the Unified Approach, is a Frequentist method with correct 
coverage and automatic transition from two-sided confidence intervals to upper limits in the case 
of negative results. 
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fluctuation has been calculated in order to obtain a band as symmetric as possible around 
// ep . In practice this is done by a computer program that simultaneously decreases the lower 
limit /iup^fe, a, 7) of the band (lower dashed lines in Fig. |3|) and increases the upper limit 
jiiup (b, a, 7) of the band (upper dashed lines in Fig. |) until the condition 

n( + ) (6,0,7) 

J2 P(n\ f i = 0,b)> 1 (2.5) 

n=rv-~ )(6,a,7) 

is reached. Here rS~\b, a, 7) and n^ + \b, a, 7) are the values of n such that 

/i up (n H (b,«,7),&,«) = fj^(b,a,i) , /i up (n (+) (fr, a, 7), 6, a) = /i^ } (6,a,7) . (2.6) 



The inequality sign in Eq. ( |2.5| ) is needed because n is an integer and in general it is not 
possible to obtain exactly the desired probability 7. This is also the reason for the fact that 
the dashed lines in Fig. |3| are not smooth. The dotted lines in Fig. |3] represent the lower 
limit for fi U p{n, b,a = 0.90) as a function of b, that is obtained for n = 0. 

It is clear that having analyzed the data using, for example, the Unified Approach and 
having obtained an upper bound /i up with an expected background b, looking at Fig. |]A one 
can judge if the upper bound fi up is reasonable or the result of an unlikely statistical fluc- 
tuation. For example, the Heidelberg-Moscow double-beta decay experiment |H| measured 



recently n = 7 events with an expected background b = 13. Using the Unified Approach they 
obtained /i up = 2.07 at 90% CL, with exclusion potential fi ep = 7.51. Looking at Fig. [3]A 
one can see that the 90% lower limit for /i up assuming jj = is /i U p^ = 2.07, so the discrep- 
ancy between ^ up and fi ep is just acceptable at the border of 10% probability. Using the 
Bayesian Ordering method they would have obtained n U p{n = 7, b = 13, a — 0.10) = 3.33, 
with exclusion potential /%,(& = 13, a — 0.10) = 8.11, and with the Bayesian Theory (with 
a flat prior and shortest credibility intervals) fi up (n = 7, b = 13, a = 0.10) = 4.01 and 
//ep(6 = 13, a = 0.10) = 7.81. 

Of course, one can obtain the same results using the measured value of n and the Poisson 
p.d.f. P(n|/i = 0,6) with the expected background b. The advantage of Figs. [I] and ^] is 
that one can easily transform fi up and /x ep in the physical quantity of interest, presenting the 
result only in terms of physical quantities. For example, in double-beta decay experiments 
H is connected to the effective Majorana mass | (m) | of the electron neutrino by the relation 

I0»>l = f$|. (2-7) 

where £ is a constant with dimension of mass that depends on the decaying nucleus and 
A4 is the nuclear matrix element. In the case of the Heidelberg- Moscow double-beta decay 
experiment fT5| , the decaying nucleus is 76 Ge, with £ = 0.57 eV. Using the nuclear matrix 



element \JA\ = 2.80 calculated in [pT6| , the 90% CL upper bound for the effective Majorana 
mass obtained with the Unified Approach is |(m)| up = 0.29 eV, the corresponding exclusion 
potential is |(m)| ep = 0.56 eV and the 90% lower limit for the fluctuations of |{m)| up (if 
I (to) I = 0) is |(m)| U p^ = 0.29 eV. Using instead the Bayesian Ordering method, we obtain 
I (to) | U p = 0.37 eV, |(m)| ep = 0.58 eV and |(m)| up ^ = 0.37 eV, and with the Bayesian Theory 
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(with a flat prior and shortest credibility intervals) we have |(m)| up = 0.41 eV, \{m)\ 



(-) 

up 

Let us define the Pull of a null result as 



cp 



0.57 eV and |(m)| up ; = 0.41 eV. 



Pull(n, b, a) = ^M-^a) 

If Pu11(t2,, b, a) > 1, the experimental upper limit // up is significantly weaker than the exclu- 
sion potential /i ep and may be considered as a weak indication that a signal may be present 
(// > 0). On the other hand, if Pull(n, b, a) < —1 and there is no doubt on the value of the 
mean background b, it means that the experiment has experienced an unlikely low fluctua- 
tion of the background and the resulting upper bound, that is significantly more stringent 
than the exclusion potential, is not reliable from a physical point of view and it is likely 
to increase if the experiment is continued (a quite undesirable behavior), as shown by the 
example of the KARMEN experiment (see Fig. 4 of Ref. [0). Moreover, a method that 
gives values of the Pull closer to zero produces upper bounds that are more reliable from a 
physical point of view. 



For example, in the case of the Heidelberg-Moscow double-beta decay experiment |To 
we have n = 7, b = 13 and a = 0.10, giving a cp = 3.91 and Pull = —1.39 in the Unified 
Approach, a ep = 3.52 and Pull = —1.36 with the Bayesian Ordering method, and a cp = 2.98 
and Pull = —1.27 in the Bayesian Theory with a flat prior and shortest credibility intervals. 
Hence, among the two Frequentist methods that we have considered, the upper bound 
obtained with Bayesian Ordering is slightly more reliable, from a physical point of view, 
than the one obtained with the Unified Approach. If one is willing to accept the Bayesian 
Theory, the corresponding upper bound is clearly the most reliable one from a physical point 
of view. 

Let us point out that the knowledge of the possible fluctuation of the upper bound // up 
with respect to the exclusion potential also help to decide, before looking at the data, 

which is the most appropriate Frequentist method for the statistical analysis. 

This can be understood comparing, for example, Figs. |3|A and |3|B, obtained with two 
Frequentist methods with correct coverage. One can see that the upper dashed lines in 
the two figures almost coincide and the exclusion potential is slightly lower in the Unified 
Approach, with a difference going from 0.30 for b = 1 to 0.73 for b = 19, with a relative 
difference of 8-9%. On the other hand, the lower dashed lines and the dotted lines obtained 
with the Bayesian Ordering are significantly higher than those obtained with the Unified 
Approach. The difference of the lower dashed lines, goes from 0.43 for b = 1 to 1.40 for 
b = 19, with a relative difference going from 27% to 64%. The difference between the lowest 
possible values of /t up (dotted lines) is quite large: for b > 4 the lowest possible values of 
// U p is about 0.8 in the Unified Approach and about 1.8 in the Bayesian Ordering method, 
more than twice! 

Hence, if one does not want to risk to have to present a very stringent limit, which 
would be statistically correct but physically misleading, in the case of observation of less 
events than the expected background, one can look at figures like Figs. |3|A and |3]B and 
the corresponding ones for other Frequentist methods and decide which is the method more 
suitable for his tastes. Let us emphasize that this choice must be done before looking at the 



6 



data. If one chooses the statistical method on the basis of the data, the property of coverage 
is lost. 

Furthermore, the exclusion potential of an experiment can be calculated and published 
before starting the experiment or before the data are known, in order to have an indication 
of the excluded region that will be obtained in the case of absence of a signal (or a signal 
much smaller than the expected background). We think that it would be useful to publish 
together with the exclusion potential also the standard deviation of the upper limit in the 
absence of a signal, in order to illustrate the possible fluctuations of the excluded region and 
to give, at the same time, a quantitative statement on the precision of the experiment. 



III. DETECTION FUNCTIONS 

The exclusion potential and the standard deviation of the upper limit in the absence of 
a signal are interesting quantities, but they give only information on the possible experi- 
mental result in the worst-case scenario, that in which the signal is absent or so small to 
be undetectable. Usually researchers are more interested in finding positive signals. For 
example, they would like to know in advance which would be the most likely outcome of 
the experiment if there is a true signal. In this case, we propose to calculate the upper 
and lower detection functions, b, a) and b, a), obtained averaging the upper and 

lower limits fj, np (n,b,a) and fi\ ow (n,b,a) over n with the Poisson p.d.f. P(n|/j,&): 



/i±(/i, b, a) = fi^p (n, b, a) P(n\fi, b) . (3.1) 

n=0 

The standard deviation of // "p (n,b,a) is given by 

oo 2 

o±(ii, b, a) = } \fi ip (n, b, a) — ;li±(/^, b, a) P(n\fi,b) . (3.2) 

' J I low 



n=0 



In Fig. [| we have plotted (upper and lower solid lines, respectively) as functions of 

\x for a = 0.10 and b = 13 in the Unified Approach Jl| (Fig. §A), in the Bayesian Ordering 
method M (Fig. |]B) and in the Bayesian Theory assuming a flat prior and shortest credibility 
intervals (Fig. |4]C). The shadowed regions delimited by the dashed lines in Fig. || represent 
the bands /i + ± a + (upper band) and /i_ ± cr_ (lower band). From Fig. |] one can see that 
the average upper bound /i + is almost identical in the three methods, but the range /i + ± a + 
for small values of /i is shortest in the Bayesian Theory and largest in the Unified Approach. 
The average lower bound /i_ and the range /i_ ± cr_ are similar in the three methods, with 
the small difference that /i_ — ct_ > for /i ~ 9 in the two Frequentist methods (Unified 
Approach and Bayesian Ordering) and fi ~ 10 in the Bayesian Theory. The three upper 
plots in Fig. f| show the probability to find /ii ow in the interval /i_ ± cr_ (solid lines) and 
the probability to find /i up in the interval /i + ± a + (dashed lines). The latter probability 
is high (larger than 80%) for small values of /i, where the interval /i_ ± cr_ includes zero, 
and stabilizes around 68% for higher values of /i (the fluctuations and discontinuities of the 
probability as a function of /i are due to the discreteness of n). 
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The detection functions and the standard deviations of the lower and upper bounds 
show the expected result and its possible fluctuations if the signal under measurement is not 
negligibly small. In the next section we present a definition of sensitivity of an experiment 
to a signal. 



IV. SENSITIVITY TO A SIGNAL 

Often researchers would like to plan an experiment capable of revealing a signal whose 
value is indicated by previous measurements or predicted by theory. Hence, it is useful to 
define the sensitivity of an experiment to a signal. 

Two probabilities must be involved in the definition of the sensitivity to a signal: the 
confidence level 1 — a of the confidence interval that represents the result of the experiment 
and the probability A to find a confidence interval with a positive lower bound. 

We think that an appropriate definition of the 100A% sensitivity corresponding to a 
100(1 —a)% confidence level, fi s (b, a, A), of an experiment measuring a Poisson process with 
known background b is the value of \i for which there is a probability A to find a positive 
lower limit for ji with confidence level 100(1 — a)%. Hence, we define /i s (6, a, A) through the 
equation^ 

£ P(n\ f i s (b,a,\),b) = X, (4.1) 

n>n B (b,ce) 

where n s (b, a) is the smallest integer such that 

Vlaw(n B (b, a),b,a) > . (4.2) 

Here [i\ ow (n, b, a) is the lower limit of the 100(1 — a)% confidence interval (credibility interval 
in the Bayesian Theory) corresponding to the observation of n events. In all Frequentist 
methods with correct coverage that guarantee an automatic transition from two-sided con- 
fidence intervals to upper limits for n < b (as the Unified Approach |l| and the Bayesian 
Ordering method |]]), the acceptance interval for fi = starts at n^/i = 0,6, a) = and 
ends at n 2 (n = 0, b, a), where n 2 (/U = 0, b, a) is the smallest integer such that 

n2(fi=0,b,a) 

P(n\fi = 0,b) > I- a. (4.3) 

n=0 

Then it is clear that 



4 After the completion of this work, we have been informed that Hernandez, Nava and Rebecchi 
[jTJI defined similar criteria in order to calculate "discovery limits" in prospective studies. Our 
Eq. ( [4.1] ) coincides with their Eq. (6) with 5 = 1 — A, and our Eq. (O) corresponds to their Eq. (5) 
with e = a in the case of Frequentist methods with correct coverage and automatic transition from 
two-sided confidence intervals to upper limits for a small number of counts. 
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n s (b, a) = n 2 (fi = 0, b, a) + 1 



(4.4) 



is the smallest integer that satisfies Eq. (|4.2|) . 

Figure |5] shows the value of fi s (b,a, A) as a function of b in Frequentist methods (solid 
lines) and in the Bayesian Theory with a flat prior and shortest credibility intervals (dashed 
lines) for 1 — a = 0.90, 0.95, 0.99 and A = 0.50, 0.90, 0.99. The lines are not smooth because 
of the discreteness of n s (b, a) that causes jumps of the solution of Eq. (|4.1| ) as b varies from 
one value to another with different n B (b,a). 

The sensitivity fi s (b, a, A) provides useful information for the planning of an experiment 
with the purpose of exploring a range [/i m i n , p, max ] of possible values of fi that could be inferred 
from the results of other experiments or from theory. In order to do this, the background 
in the experiment must be small enough that the sensitivity fJ, s (b, a, A) is smaller than /i m i n . 
In this case the experiment will have probability 1 — a to obtain a correct result (i.e. a 
confidence interval that contains the true value of /i) and a probability bigger than A to 
obtain a positive lower limit, i.e. to reveal a true signal, if /i > yu s (6, a, A). The probability 
that the experiment will reveal a true signal within a correct confidence interval is larger 
the product (1 — a) A (if \i > fi s (b, a, A)). Therefore, it is desirable to have both 1 — a and A 
large. In Fig. ^ we have chosen 1 — a = 0.90, 0.95, 0.99, that are commonly used values for 
the confidence level, and A = 0.50, 0.90, 0.99. We think that 1 — a should be always chosen 
large, preferably 1 — a = 0.99, because getting a correct result is the most important thing. 
As for A, a large value is important in order to have good chances to reveal the signal (if it 
exist!). For example, having 1 — a — 0.99 and A = 0.99 gives a probability bigger than 98% 
to find a true signal within a correct confidence interval (if \x > fx s (b, a, A)). On the other 
hand, for 1 — a = 0.90 and A = 0.50 (A = 0.90) the probability to find a true signal within 
a correct confidence interval can be as low as 45% (81%). 

From Fig. [5] one can see that the sensitivity increases sub-linearly as the background in- 
creases. Since the background increases linearly with the time of data-taking, the sensitivity 
of the experiment increases sub-linearly as a function of data-taking time. Let us consider an 
experiment searching for a signal produced by a new process for which there is an indication 
from previous experiments or from theory. Since the signal, as the background, increases 
linearly with the time of data-taking, there is a time such that the signal becomes larger 
than the sensitivity and this time provides an estimate of the data-taking time necessary to 
reveal the new process. 

It is interesting to notice that the sensitivity for b = 13, 1 — a = 0.90 is /i s ~ 7 for 
A = 0.50 and [i s ~ 13 for A = 0.90, in rough agreement with the lower bands in Fig. ^, which 
show that there is a good chance to find a lower limit for \x bigger than zero if fi > 10. 

From this example it is clear that the sensitivity of an experiment is different from 
its exclusion potential. The proposals of new experiments on the search for a signal should 
present the sensitivity as the most interesting characteristic of the experiment. The exclusion 
potential should be also presented as an illustration of the potentiality of the experiment in 
the most unfortunate case of absence of a signal. 

Let us remind the reader that the word "probability" has different definitions in the 
Frequentist and Bayesian theories. In the Bayesian theory "probability" is defined as "degree 
of belief" , whereas in the Frequentist theory it is defined as ratio of the number of positive 
cases and total number of trials in a large ensemble. The Frequentist definition avoids the 
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need of subjective judgment, but it is not clear what is its meaning in the planning and 
realization of one experiment (or a few experiments). Whatever the meaning of Frequentist 
probability in this case, we think that it is comforting to see from Fig. || that the Frequentist 
and Bayesian values for fi B (b, a, A) are quite close. 

V. CONCLUSIONS 

In conclusion, we have defined some quantities that may help to asses the reliability 
(physical significance) of the confidence intervals obtained with different methods. We have 
also defined appropriately the sensitivity of an experiment to a signal. 

In Section |H we have considered the quantity called "sensitivity" by Feldman and Cousins 
and we have argued that this name is not appropriate because this quantity does not 
represent the capability of an experiment to reveal a signal. We proposed to call this quantity 
"exclusion potential". 

Considering the case of a Poisson process with known background, we have shown how 
the exclusion potential and the standard deviation of the upper limit in the absence of 
a signal may help to choose the method that is more appropriate for obtaining reliable 
upper limits (Section [TT]). We have also defined the Pull of a null result, that quantifies the 
reliability of an experimental upper limit. In Section III we have defined the upper and 
lower detection functions, that give the most likely outcome of an experiment if there is a 
signal. In Section f[V] we proposed an appropriate definition of sensitivity of an experiment 
to a signal. These definitions apply to both Frequentist and Bayesian statistical theories 
and can be easily generalized to any process in which a quantity fi with known probability 
distribution is measured: the upper (lower) detection function is the average of the upper 
(lower) limit and the 100A% sensitivity is the lower value of \x for which there is a probability 
A to find a positive lower limit. 

We considered explicitly the case of a Poisson process with known background in the 
framework of Frequentist and Bayesian statistical theories, but similar considerations and 
conclusions apply also to the case of a Gaussian distribution with boundary. 
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FIG. 1. 90% CL exclusion potential fj, ep (solid line) and fi ep ±a ep (dashed lines) as functions of 
the known background b obtained with the Unified Approach (A) Q, with the Bayesian Ordering 
method (B) || and with the Bayesian Theory (C) with a flat prior and shortest credibility intervals. 
The probability to obtain a 90% CL upper bound in the shadowed region if fj, = is shown in the 
three upper figures a, b and c. 
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FIG. 2. Probability of the possible values of the 90% CL upper bound fj, up in the absence of 
a signal (fi = 0) and a known background b = 13 in the Unified Approach (A) in the Bayesian 
Ordering method (B) Q and in the Bayesian Theory (C) with a flat prior and shortest credibility 
intervals. The dashed vertical lines show the value of the exclusion potential /i ep and the dotted 
vertical lines represent fi ep ± a cp . 
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(lO %06) dn r1 

FIG. 3. 90% CL exclusion potential /i ep (solid line) and 90% width of the 90% CL upper limit 
(shadowed region delimited by the dashed lines) obtained with the Unified Approach (A) with 
the Bayesian Ordering method (B) || and with the Bayesian Theory (C) with a flat prior and 
shortest credibility intervals. The dotted lines represent the lower limit for ^ up (90% CL), obtained 
for n = 0. 
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FIG. 4. Upper and lower 90% CL detection functions (a = 0.10) (Eq. (3.1)) ^ + (upper solid 
lines) and \i- (lower solid lines) as functions of [i for b = 13 obtained with the Unified Approach 
0] (A), with the Bayesian Ordering method (B) and with the Bayesian Theory assuming a flat 
prior and shortest credibility intervals (C). The shadowed regions delimited by the dashed lines 
represent the bands /i + ±o+ (upper band) and /x_ ±cr_ (lower band), with a± defined in Eq. ( |3.2|) . 
The three upper figures a, b and c show the probability to find an upper limit /x up in the interval 
fj, + ± cr_|_ (dashed line) and the probability to find an lower limit m ow in the interval \i- ± cr_ (solid 
line). 
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FIG. 5. Sensitivity fj, B (b,a,\) (see Eq. ([O])) as a function of b in Frequentist methods (solid 
lines) and in the Bayesian Theory with a flat prior and shortest credibility interval (dashed lines) 
for 1 - a = 0.90, 0.95, 0.99 and A = 0.50, 0.90, 0.99. 
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